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Abstract 

The orbit O of highest weight vectors in a complex simple Lie algebra q is 
the smallest non-zero adjoint orbit that is conical, i.e., stable under the Euler 
dilation action of C* on q. O is a smooth quasi-amne variety and its closure 
Cl(O) is a singular quadratic cone in q. The varieties O and Cl(O) have the 
same algebra of regular functions, i.e. R{0) = i?(Cl(C)), and hence the same 
algebra of algebraic differential operators, i.e., V(0) = V(CI(0)). 

The content and structure of T>(0) is unknown, except in the case where 
q = sp(2n,C). The Euler action defines natural g-invariant algebra gradings 
R{0) = (B p&+ R p (0) and V(0) = ® V& V V (0) so that D{R q (0)) C R q+P (0) 
if D G V p (0). 

We construct, for q classical and different from sp(2n,C), a subalgebra A 
inside V{0) which, like R(0), is g-stable, graded and maximal commutative. 
Unlike R(0), which lives in non-negative degrees, our algebra A lives in non- 
positive degrees so that A = ® p ^z + A- p . The intersection A fl R(0) is just 
the constants. The space A-i is finite-dimensional, transforms as the adjoint 
representation and generates A as an algebra. Moreover we construct a q- 
equivariant graded algebra isomorphism R(0) — > A, f i— ► Dj. The differential 
operators in A-\ have order 4. 

The operators in A are "exotic" in that they lie outside the realm of familiar 
differential operators. Although their existence is predicted (as a conjecture) 
by the quantization program for nilpotent orbits of the second author, our 
actual construction, by quantizing "exotic" symbols we obtained previously, is 
quite subtle. 

We will use the operators in A-i to quantize O in a subsequent paper 
and to construct an algebraic star product on R{0). To pave the way for the 
quantization, we show that the formula (f\g) = (Dg /)(0) defines a positive- 
definite Hermitian inner product on R(0). 

1991 Mathematics Subject Classification: 16S32, 58F06, 81S10, 14L30 
*Department of Mathematics, UC Davis, E-mail address: ast@math.ucdavis.edu 
^Department of Mathematics, Perm State University, E-mail address: rkb@math.psu.edu 
tResearch supported in part by NSF Grant No. DMS-9505055 



1 



1 Introduction 



The nature of the algebra T>(X) of differential operators on a complex quasi-projective 
algebraic variety X is rather mysterious. There is only one rather general result: if 
X is both smooth and affine, then (i) T>(X) is generated by the functions and vector 
fields on X, and (ii) every function on T*X which is a homogeneous polynomial on 
the fibers of T*X — > X is the principal symbol of a differential operator. As soon as 
X becomes non-smooth or non-affine, the generation quickly fails and the surjectivity 
of the symbol map is not well understood. 

Let X be a homogeneous space of a reductive complex algebraic group K. Then 
X is a quasi-projective complex algebraic manifold. Differentiation of the i^-action 
gives an infinitesimal action of the Lie algebra t on X by vector fields. This gives 
rise to a complex algebra homomorphism 

U(t) - V{X) (1) 

where is the universal enveloping algebra of t. 

Suppose X is compact, i.e., projective. Then W. Borho and J.-L. Brylinski 
( ||Bo-Br |) proved that the homomorphism ([j]) is surjective. They also computed 



its kernel which is a primitive ideal in U(t). Here X is a (generalized) flag variety 
K/Q. These spaces X occur as the projectivized orbits of highest weight vectors 
in finite-dimensional irreducible representations of K. We note that K admits only 
finitely many compact homogeneous spaces. These orbits play a key role in algebraic 
geometry, symplectic geometry and representation theory. 

There are natural non-compact analogues to the flag varieties. These homoge- 
neous spaces of K arise in the following way. Suppose that K sits inside a larger 
reductive complex algebraic group G as (the identity component of) a spherical sub- 
group. Let g = © p be the corresponding complex Cartan decomposition where 
t = Lief^ and g = LieG. Then the i^-orbits in p are quasi-affine homogeneous 
spaces X of K (and so non-compact except if X is a point). The choice G = K x K 
gives rise to all the adjoint orbits of K. 

Among all i^-orbits in p, there are only finitely many orbits Y which are stable 
under the Euler dilation action of C* on p. These orbits are exactly the ones consist- 
ing of nilpotent elements of g; accordingly they are called "nilpotent orbits". There 
is a rich literature on the geometry of these orbits Y, especially with respect to the 
quantization of the corresponding (see |j5e| ) real nilpotent orbits. The most recent 
development here is the Kaehler structure on real nilpotent orbits arising from the 
work of Kronheimer (|Kr|) and Vergne (QV§). See also pijl, [|B2], [B5|. 

The Euler action on Y provides f^-invariant algebra gradings R(Y) = ® p ^zR p {Y) 
and T>(Y) = @ P ezV p (Y) where R P (Y) is the subspace of Euler homogeneous func- 
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tions of degree p and 

V p (Y) = {D G V{Y) | D(R q (Y)) C R q+P (Y) for all q G Z} (2) 

We say that differential operators in T> P (Y) are Euler homogeneous of degree p. The 
vector fields of the infinitesimal 6-action are Euler homogeneous of degree since the 
.fT-action commutes with the Euler action. 

T>(Y) is not, in general, generated by the global functions and vector fields on Y. 
We can see this already in the simplest case, namely the case where g = sp(2n, C), 
6 = gl(n, C) and Y is a minimal (non-zero) orbit in p. (So Y is the orbit of highest 
weight vectors in p + or p~.) Then Y is isomorphic to the quotient (C n — {0})/Z 2 
where Z 2 acts by ±1. It is easy to see that T>(Y) is the even part of the Weyl algebra, 
i.e., 

T>(Y) = C[xi, . . . , x n , J^-, . . . , ^-] Z2 = C[xiXj,Xi^j, d ®. 9x . ] (3) 

Clearly the subalgebra generated by the functions and vector fields is C[xj,Xj, a^J^r]- 
This subalgebra lives in non-negative degrees, and so does not contain the operators 
= a ®.Q x . - The operators Dij generate a maximal commutative subalgebra A of 
T>(Y) which lives in non-positive degrees. 

In most cases beyond the example above, it is a hard open problem to determine 
T>(Y) and to construct any "new operators" in it beyond the known operators, i.e., 
those generated by functions and by the vector fields of the g-action. See Section |5| 
for a few known examples where highly non-obvious "new " operators in T>(Y) have 
been constructed. 

We expect T>(Y) to contain "new" differential operators for general nilpotent 
orbits Y (cf. P3|| ). These will be "exotic" in the sense that they lie outside the 
realm of familiar differential operators. 

In this paper, we study the case where Y is the complex minimal nilpotent orbit 
O of a simple complex Lie group G. (This is the case where we embed if as a 
spherical subgroup of K x K ; then we rename K as G.) See [0 and ||B-K1| for some 
results on the geometry and quantization of O. 

The space O is just the orbit of highest weight vectors (or equivalently, of highest 
root vectors) in g. Accordingly O has a number of very nice properties. The closure 
of O is C\(0) = O U {0} and C\(0) is a quadratic cone in g, i.e., C\(0) is cut out 
(scheme-theoretically) by homogeneous quadratic polynomial functions on g. All 
regular functions on O extend to C\{0). So O and Cl(O) have the same algebra of 
regular functions, i.e., R{0) = R(C\(0)), and hence the same algebra of algebraic 
differential operators, i.e., T>{0) = V(C\(0)). Notice that O "misses" being a smooth 
afline variety by just one point, namely the origin. 

The algebra R{0) is graded in non-negative degrees so that R{0) = ® p ^z + R p {0). 
The subspace R\{0) generates R{0) as an algebra and consists of the functions 
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f x , x 6 Q, obtained by restricting the linear functions on g. So f x is defined by 
f x (z) = (x, z) g where (•, -) is the (normalized) Killing form of g. 

Differentiation of the G-action on O gives an infinitesimal vector field action 

-> 5JectC, x i-> rf (4) 

These vector fields vf are Euler homogeneous of degree 0. The subalgebra of T>(0) 
generated by R(0) and by the rf is thus Euler graded in non-negative degrees. 

We assume that g is classical in order to prove our results. We exclude the case 
g = sp(2n, C), n > 1, as then O ~ (C 2n - {0})/Z 2 and so V(0) is just the even part 
of a Weyl algebra. 

We construct inside T>(0) a G-stable, graded, maximal commutative subalgebra 
A isomorphic to R(0). Unlike R(0), which lives in non-negative degrees, our algebra 
A lives in non-positive degrees so that 

A = A^ p (5) 



where A- p = A n V_ p {0). The intersection A n R{0) is just A = R {O) = C. 
The space A-i is finite-dimensional, transforms as the adjoint representation and 
generates A as an algebra. We construct a G-equivariant graded algebra isomorphism 
ll(O) >A. /• > I) f . 

The differential operators in A-i have order 4 and are "exotic" in the sense 
discussed above. We construct A-i by quantizing a space r(g) C R{T*0) of symbols 
which we obtained in [ A-B2 |. We next recall the context in which we found r(g). 



In ||A-B1|| , we constructed complex Lie algebra embedding 

t:q®Q^C(T*0) (6) 

where C(T*0) is the complex Poisson algebra of rational functions on T*0. We did 
this in a more general context which includes the cases Y = O. 
The restriction of r to the diagonal in g © g is the natural map 

$ : g -> R(T*0), x i-> (7) 

where $ x is the principal symbol of the vector field if. I.e., t(x, x) = <3> x . On the 
anti-diagonal r is given by 

t(x, -x) = f x - g x (8) 

where g x , x G g, are rational functions on T*(9 defined on a G-invariant (Zariski) 
open algebraic submanifold {T*0) re9 . 
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The functions g x transform in the adjoint representation of G. Each function g x 
is homogeneous of degree 2 on the fibers of T*0 — > O. Also g x is homogeneous of 
degree —1 with respect to the canonical lift of the Euler scaling action on O. The 
functions g x , like the functions f x , Poisson commute; i.e., {f x , f y } = {g x ,g y } = for 
all x, y G 0. 

In [ A-B2 , we computed g x for a restricted class of orbits Y which still includes 



Y — O. Let A G R(T*0) be the principal symbol of the Euler vector field E on O. 
We found that {T*0) re9 is the locus in T*(0) where A is non- vanishing. We got the 
formula: 

9* = % (9) 

where 

r : -> R(T*0), x h-> r x (10) 

is an explicit complex linear g-equi variant map. 

Each function r x is a homogeneous degree 4 polynomial on the fibers of T*(0) — > 
(9. Also r x is homogeneous of degree —1 with respect to the canonical lift of the 
Euler action. The functions r x again Poisson commute, i.e., {r x ,r y } = 0. 

We say that a function G R(T*X) which is homogeneous of degree d on the 
fibers of T*X —>■ X (and hence a polynomial function on the fibers) is an order d 
symbol. The order d symbol map sends T> d (X) into R d (T*X) where T> d (X) is the 
space of differential operators of order at most d and R d (T*X) is the space of order d 
symbols. We say that <fi G R d (T*X) quantizes into D if D G T> d (X) has order d and 
the principal symbol of D is 0. Unless X is smooth and affine, there is no guarantee 
that a given symbol will quantize. 

In this paper we quantize the symbols r x into order 4 differential operators D x 
on O in a manner equivariant with respect to both the G-action and the Euler en- 
action. We show that this equivariant quantization is unique. In our next paper 



A-B3|| , we use these same operators D x to quantize O by quantizing the map (gj). 
We obtain a star product on R{0) given by "pseudo-differential" operators. 

We construct the operators D x by manufacturing a single operator D = D xo G 
I>ii (0) where x G is a lowest weight vector. Here V*(0) = V d (0) n V p (0). We 
build D so that its principal symbol is r = r xo and D is a lowest weight vector for 
a copy of in T>^ 1 (0). Then all other operators D x are simply obtained by taking 
(iterated) commutators of Dq with the vector fields if . 



To construct Dq, we use our explicit formula for ro from [|A-B2|] to quantize tq into 



a differential operator on a (Zariski) open set O reg in O. Our quantization method is 
a modification of Weyl quantization by symmetrization. We are forced to modify the 
quantization of a factor A 2 appearing in a term of ro in order that the quantization 



5 



of r is g-finite and hence extends to an operator on all of O. See Subsection [O] for 
a detailed discussion of our strategy. 

In essence, we are adding lower order correction terms to the true Weyl quanti- 
zation of r in order to obtain D . The correction is uniquely determined, but its 
nature is mysterious to us. 



This paper can largely be read independently of ||A-B1|| and ||A-B2||, as the symbols 



in fact only motivate the construction of the differential operators. Once we figure 
out the correct formula for D , we give a self-contained proof that D 6 
The more abstract and general results we prove in Section ||] for differential operators 
on cones of highest weight vectors then give in particular the main properties of 
our operators D x : (i) the operators D x commute, (ii) the operators D x generate a 
maximal commutative subalgebra of T>(0), and (iii) f x and D x are adjoint operators 
on R(0) with respect to a (unique) positive definite Hermitian inner product (-|-) 
on R(0) such that (1|1) = 1. 

A basic question in quantization theory is whether one can set up a preferred 
quantization converting functions on T*X which are polynomial on the fibers of 
T*X — > X into differential operators. This certainly involves quantizing symbols into 
operators which include "lower order" terms. This has been studied in a different 
context in ||L-0|| - see Section |. 

In Section [5] we compare our constructions with the results of Levasseur, Smith 
and Stafford on the Joseph ideal and on differential operators on classical rings of 



invariants, and with the constructions in |B-K2 



2 Differential operators on conical orbits 
2.1 Differential operators on G- varieties 

Let X be a complex algebraic quasi-projective variety. Let T>x be the sheaf of alge- 
braic differential operators on X. Then T>(X) = T(X,T>x) is the algebra of (global) 
differential operators on X. Let T> d (X) C T>(X) be the subspace of differential oper- 
ators of order at most d. See e.g. [|B-K2| , Appendix] for some of the basic definitions 
and facts on differential operators. 

Let R(X) denote the algebra of regular functions on X. For any commutative 
complex algebra A, let D(A) be the algebra of differential operators of A. We have a 
natural algebra homomorphism T>(X) — > D(R(X)) and this is an isomorphism if X 
is quasi-affine. We will be dealing primarily with differential operators on quasi-affine 
varieties. 

Throughout the paper, we are working with complex quasi-projective algebraic 
varieties. The functions, vector fields, and differential operators we deal with are all 
regular, or algebraic, in the sense of algebraic geometry. 
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Our main emphasis is on working with smooth varieties, which are then complex 
algebraic manifolds and admit cotangent bundles. We often use the term "algebraic 
holomorphic" in place of "regular" to emphasize that the algebraic structure is an 
overlay to the underlying holomorphic structure. 

Now suppose X is smooth. Let G be a reductive complex algebraic group. Assume 
G acts on X by an algebraic holomorphic group action. Then there are induced 
representations of G on R(X) and on T>(X) by (complex) algebra automorphisms. 
Both G-representations are locally finite and completely reducible. 

We use the term G-linear map to mean a G-equivariant complex linear map of 
vector spaces. A vector space V is G -irreducible if V is an irreducible representation 
of G; often V will be a G-stable space inside a larger representation. If V is G- 
irreducible and W is some G-representation, then we say that a non-zero G-linear 
map V — > W gives a "copy of V inside W" . 

The differential of the G-action on X is the infinitesimal algebraic holomorphic 
vector field action 

g^WtctX, x^rf (11) 

where g = Lie(G) and rf z = ^|i =0 (exp — tx) ■ z. These vector fields give a complex 
Lie algebra homomorphism 

g^EndP(A), x i-> ad if (12) 

where a.dr] x (D) = [rj x ,D]. This representation of g on T>(X) extends canonically to 
a representation of q on T>(X') where X' is any Zariski open set in X. 

Proposition 2.1.1 Suppose G acts transitively on X . Let D be a differential oper- 
ator on a Zariski open set in X . Then D extends to a differential operator on X if 
and only if D is finite. 

Proof: We have a filtration of T>x by the sheaves T* k x of differential operators on X 
of order at most k. For each k, T> k x is the sheaf of sections of some complex vector 
bundle on X of finite rank. So the result follows by the same general lemma we used 
A-B2|| to prove the corresponding statement for R(T*X). □ 
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We say that X is a G-cone if there is a (non-trivial) algebraic holomorphic group 
action of C* on X commuting with the G-action. The infinitesimal generator of the 
C*-action is the algebraic holomorphic Euler vector field E on X. The Euler action 
defines G-invariant algebra gradings 

R(X) = @R P (X) and V(X) = @V p (X) (13) 
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where 



R p (X) 
V P {X) 



{feR(X)\E(f)= P f} 
{DeV(X)\[E,D]=pD} 



Then, if X is quasi-affine, 



V P (X) = {De V(X) I D(R g (X)) C R P+q (X) for all q E Z} 



(14) 



The Euler grading of T>(X) is compatible with the order filtration so that we get 
a grading of the space of differential operators of order at most d 



where V d = V d nV p . 

2.2 Differential operators on the cone of highest weight vec- 



In this subsection, we assume that X is the G-orbit of highest weight vectors in a 
(non-trivial) finite-dimensional irreducible complex representation V of G. A (non- 
zero) vector v in V is called a highest weight vector if v is semi-invariant under some 
Borel subgroup B of G. (In Cartan-Weyl highest weight theory, one usually fixes a 
choice of B, and then there is, up to scaling, exactly one B-semi- invariant vector. 
The conjugacy of Borel subgroups implies that the set of all highest weight vectors 
is an orbit.) 

X is a locally closed subvariety of V and is stable under the scaling action of C* 
on V. So X is a G-cone with Euler vector field E. In fact the quotient X/C* is the 
unique closed G-orbit in the projective space P(V). The quotient X/C*is often called 
the projectivization P(X) of X. The closure C\(X) of X in V is equal to X U {0} 
and X is the unique non-zero minimal conical G-orbit in V. 

The orbit X has several very nice properties ( ||V-P|| ): 

(i) the closure C1(X) is normal. 

(ii) the graded G-equivariant algebra homomorphism R(C\(X)) — > R(X) given by 

restriction of functions is an isomorphism. 

(iii) the algebra R(X) is multiplicity-free as a G-representation and R P (X) is G- 
isomorphic to the pth Cartan power of V*. 



V d = Q)V d p 



(15) 



tors. 



S 



These properties of X can be derived using the Borel-Weil theorem on P(X) . 

If a G-irreducible vector space W carries the G-representation V^, then the pth 
Cartan power ofW is the (unique) G-irreducible subspace W Up of the pth symmetric 
power S P (W) which carries V PfJj . Here V u denotes the finite-dimensional irreducible 
G-representation of highest weight v. Geometrically, if W is the space of holomorphic 
sections of a G-homogeneous complex line bundle C over P(X), then W Up identifies 
with the space of holomorphic sections of 

The algebra R(V) identifies with the symmetric algebra S(V*). Properties (ii) 
and (iii) say that the natural graded algebra homomorphism 

C : S(V*) - R(X) (16) 

defined by restriction of polynomial functions from V to X is surjective and the pth 
graded map ( p : S P (V*) — > R p (X) induces a G-linear isomorphism ( p : (\/*) Sp — > 
R p (X). 

Consequently, R P (X) = for p < and 

Ro(X) = C (17) 

Also R(X) is generated by Ri(X). In degree 1, ( gives the G-linear isomorphism 

V*^R 1 (X), a^f a (18) 

where f a (w) = a(w). Also property (ii) implies that 

V{X) =V(Cl(X)) (19) 

We have a natural graded algebra inclusion of R{X) into T> (X) as the space of 
order zero differential operators; so R P (X) C T> p (X). Moreover R(X) is a maximal 
commutative subalgebra of T>(X) living in non-negative degrees. A natural question 
is to find other maximal commutative subalgebras, and to see what degrees they live 
in. 

To figure out what we might expect to find, let us consider the simplest example. 
This occurs when V = C N is the standard representation of G — GL(N,C) and so 
X = C N - {0}. Then R(X) = R(C N ) and so we get the Weyl algebra 

V(X) = V(C N ) = CK, ...,„„,!_,...,!-] (20) 

In addition to R(X) = C[vi, . . . ,v n ], we have a second "obvious" maximal commu- 
tative subalgebra of V(X), namely the algebra 

«4 = C[Jb---,fc] (21) 
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of constant coefficient differential operators. A lives in non-positive degrees. The 
assignment Vi i— > J^- defines a graded algebra isomorphism R(X) — > A. A is gen- 
erated by the space A-± of constant coefficient vector fields and A- p ~ V Up as 
G-represent at ions . 

We see no obvious way to generalize the construction of L = A-i in the example 
above to the case where V is arbitrary. However, if one has such a space L then 
we can prove the following. This abstract result gives us the commutativity of the 
concrete exotic differential operators we construct later in Section |j. 

Proposition 2.2.1 Let X C V and X* C V* be the G-orbits of highest weight 
vectors. Suppose we have a finite- dimensional G-irreducible complex subspace L C 
T>_i(X) which carries the G -representation V . Then the differential operators D G L 
all commute and hence generate a commutative subalgebra A C T>(X). Consequently 
A = ® p( zi + A-p is graded in non-positive degrees with Aq = C and A-\ = L. 

A is isomorphic to R(X*). In fact, if we fix a G-linear isomorphism V — > L, 
v i — > D v , then the map R\{X*) — > A-i, f v >— > D v , extends uniquely to a graded 
G-linear algebra isomorphism R(X*) — > A, f \—> Df. 

Proof: Let us consider the complex G-linear map 7r : A 2 V — > Endi?(X) given by 
ir(uAv) = [D u , D v ]. The image of ti lies in the space End[_ 2 ]-R(X) of endomorphisms 
which are homogeneous of degree —2 with respect to the Euler action. So n is just 
a collection of complex G-linear maps 

ti p : AY -> Rom(R p (X), R P - 2 {X)) (22) 

Thus tt p is an intertwining operator between two G-representations, namely A 2 V and 
Hohi((V*) h p, (y*) H (p- 2 )) ~ Hom(^ H (P- 2 ), V Up ). 
But it follows from highest weight theory that 

Hom G (AV, Hom(V H(p - 2) , V Up )) = 0. (23) 

Indeed, let v be the highest weight of V so that V uk ~ Vkv Every irreducible G- 
representation occurring in A 2 V has highest weight A < 2v by Cartan product theory. 
On the other hand, suppose /i is the highest weight of some irreducible representation 
occurring in Hom(V( p _2)^, V pu ). The theory of Parasarathy, Rao and Varadarajan for 
tensor product decompositions gives here that fi > — (p — 2)v +pv = 2v. Thus A = fi 
is impossible. This proves (p3|) . 

Therefore tt p = for all p and so it = 0. This proves the commutativity of the 
operators D v . The chosen map V — > L defines a surjective G-linear graded algebra 
homomorphism S(V) — > ^4. The image of 5 p (y) is the graded component y4._ p . 
Clearly A- p ^ for p > 0. 



10 



We claim that the two graded algebra homomorphisms S(V) — > A and S(V) — > 
R(X*) have the same kernel. Indeed, suppose a G-irreducible subspace W C A- p 
carries the representation with highest weight //. Then arguing as above, we find 
that ji < pv since W occurs in S P (V) and also // > pv because each operator 
A G A- p is a collection of maps Aj : Rj(X) — > Rj_ p {X). This forces \i = pv. But 
the representation V pu occurs exactly once in S P V, as the Cartan piece V Up . We 
conclude that the space V Up maps isomorphically to A- p . The claim now follows. □ 



Remark 2.2.2 (i) Proposition |2.2.1| was established in ||B-K2| , Theorem 3.10] for a 



restricted class of cases (53, V). We have essentially just rewritten the proof given 
there in our more general situation. 

(ii) Let I C S(V) be the graded ideal of polynomial functions vanishing on X*. Then 
we have a direct sum S P (V) = V Up © I p . An important property of X* is that / 
is generated by its degree two piece I<i- This is a result of Kostant; see |par|| for a 



write-up. This gives another way to prove the second paragraph of Proposition 2.2.1 



(iii) Suppose L is as in Proposition [2.2. 1| but L carries the representation V* instead 
of V. Then we can show by a similar argument that V ~ V* and so the conclusions 
of the Proposition still follow. 



Proposition [2.2. 1| leads us to pose the question: in what generality does the space 
L exist? In [B-K2| , L was constructed for a restricted set of cases of highest weight 
orbits (associated to complex Hermitian symmetric pairs). The main result of this 
paper is to construct L for the minimal nilpotent orbit O in a classical complex 
simple Lie algebra different from 5p(2n, C). 

Our final result of this subsection says, under a mild hypothesis, that the com- 



mutative subalgebra A in Proposition |2.2.1| is maximal and the operators Df define 



a non-degenerate inner product on R(X). We will see later that our operators on O 
satisfy the hypothesis and the resulting inner product is positive definite. 

Let U cGbea maximal compact subgroup of G. Then V admits a [/-invariant 
positive-definite Hermitian inner product (-|-) (unique up to a positive real scalar). 
Let V denote the complex conjugate vector space to V. Then the complex conjuga- 
tion map V ^> V, v t-^v, is C-anti-linear. Each vector v G V defines a C-linear func- 
tional by on V by by(u) = (u\v). This gives a [7-equivariant identification V — > V*, 
v 1 — > b^, of complex vector spaces; we will use this identification freely from now 
on. This induces a graded C-algebra identification R(X) = R(X*). Then we get a 
graded C-anti-linear algebra isomorphism R(X) — > R(X*), g >-*g, defined in degree 

^ t*y fv 1 ^ fv = fv 



Proposition 2.2.3 Suppose we are in the situation of Proposition \2.2.1\ Let us 
assume that for each positive integer p there exists an operator D G L such that D 
is non-zero on R P (X). Then 
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(i) The algebra A, like R(X), is a maximal commutative subalgebra ofV(X). 

(ii) R(X) admits a unique U -invariant non- degenerate Hermitian inner product (-|-) ; 
such that (1|1) = 1 and, for all v G V, the operators f w and D v are adjoint on R(X). 
Then for any f,g<E R(X) we have 

(f\g) = constant term of D-g(f) (24) 

Each h G R(X) can be uniquely written as a finite sum h = J2 p ez + where h p G 
R P (X). Then, in view of fllTf ), we call ho the constant term of h. 

Proof: We see that (ii) implies (i) since R{X) is a maximal commutative subal- 
gebra of V(X). To prove (ii), we need to construct B = (-|-). 

We start by taking any [/-invariant positive-definite Hermitian inner product Q 
on R{X) such that Q(l, 1) = 1. Clearly Q exists and R P (X) and R q (X) are Q- 
orthogonal if p ^ q (since they carry different irreducible [/-representations). Thus 
Q is given by a family, indexed by p, of inner products Q p on R p (X). Similarly B 
must be given by a family B p . 

To start off, we put B = Q . Now we proceed by induction and define B p+ i by 
the relation 



B p+1 (h,f w g)=B p (D v (h),g) (25) 

where g G R P (X) and h G Rp+i(X). This relation is exactly the condition that 
multiplication by f„ is adjoint to D v . 

We need to check that B p+ \ is well-defined. Clearly the functions fag span 
R p+ i{X). Also there exists a complex scalar c p+ \ such that 

Cp+iQ p +\(h,f v g)=B p (D v (h),g) (26) 

To see this we observe that the two assignments v ®~g <8> h \— > Q p+ i(h, fag) and 
v <g) g (g) h i— > B p (D v (h), g) both define [/-equivariant complex linear maps 



Ri(X) ® Rp(X) ® i? p+ i(X) -> C (27) 

But the space of such maps is 1-dimensional - this follows using highest weight theory 
as in the proof of Proposition |2.2.1| . So B p+ i = c p+ \Q p+ \. Thus B p+ i is well-defined. 
(This was the same proof given in |B-K2j , Theorem 4.5].) 

Our hypothesis that some D v is non-zero on R p+ i(X) ensures that c p+ i ^ 0. 
Hence B p+ i is non-degenerate. Finally fl2"4]) follows as (f\g) = (Dg(f)\l). □ 
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2.3 Complex minimal nilpotent orbits 



^From now on, we assume that G is a complex simple Lie group. Then the Lie 
algebra g = Lie G is a complex simple Lie algebra and so the adjoint representation 
of G on g is irreducible. We put V = g and identify V ~ V* in an G-linear way 
using the Killing form. 

The G-orbit X of highest weight vectors in g is now the minimal (non-zero) 
nilpotent orbit O in g. Indeed, we can fix a triangular decomposition g = n + © f) ©n~ 
where f) is a complex Cartan subalgebra and n + and n~ are the spaces spanned by, 
respectively, the positive and negative root vectors. Let ip e f)* be the highest 
(positive) root and let if,/,, Z^, be the corresponding root triple so that G \) 
is semisimple and Z±^ e n are nilpotent. Then Z^, is a highest root vector in g and 
also a highest weight vector in g. So g = and X is equal to the G-orbit O of Z^. 

We choose a real Cartan involution a of g such that f) is cr-stable. Then the 
cr-fixed space in g is a maximal compact Lie subalgebra u. Then g = u + iu and a is 
^-invariant and C-anti-linear. 

We can rescale the triple (Z^, H^, Z-^) so that a(H^) = —H,p, a{Z^) = — Z_^, 
a(Z_^) = —Z^, and the bracket relations [Z^,Z-^\ = H^, [H^,Z^] = 2Z^ and 
[H^,Z_^\ = —2Z_^ are preserved. Let (•, -)s ^ e the complex Killing form on g 
rescaled so that (Z^, Z_^) s = |. We have a positive definite Hermitian inner product 
(■|-) on g defined by (u\v) = — (u,cr(v)) g . 

Let Qk be the fc-eigenspace of ad h on g where we set 

/i = ify (28) 

Since ft, is semisimple, g decomposes into the direct sum of the eigenspaces g^. It is 
well known that the eigenvalues of ad if,/, lie in {±2, ±1, 0} and g± 2 = CZ±^. Thus 
we get the decomposition 

g = g 2 ©gi ©go ©g-i ©g- 2 (29) 

We have dimg fc = dimg_ fc and the spaces q±i are even dimensional. We put m = 
|dimg ±1 . Then (see, e.g., ||B-K3|| ) 

dim O = 2m + 2 (30) 

We now put 

Xip = Zj/j, x' = H^, xq = Z-t/j (31) 

Then 

(x \x ) = 1/2 and [x' , x ] = -2x (32) 
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By [|A-B2 |, we can find a complex basis x±, . . . , x m , x[, . . . x' m of g_i such that 

(xi\xi) = = 1/2 

(33) 

— — and — 6{jXq 



We have, as in (|T8|), a G-linear isomorphism 

g^R^O), x^f x (34) 

where f x (y) = (x,y) s . Also as in QTTj ) there is an infinitesimal vector field action 

0^23ect0, x^rf (35) 

defined by differentiating the G-action on O. Then rf(f v ) = f\x,v\- We have the 
complex algebra gradings 

R(0) = R p {0) and V(0) = ($V P (0) (36) 



defined by the Euler C*-action as in (O), since R p (0) = for p < 0. 

All our constructions on (9 thus far have been algebraic holomorphic. However 
now we introduce complex conjugation on O into the picture. (See e.g. [ |B1| , Ap- 
pendix, A. 5] for a discussion of complex conjugation on varieties.) 

The map a : g — > g is a C-anti-linear involution of g preserving O. It follows that 
cr, as well as —a, is an anti-holomorphic involution of O which is also real algebraic. 
We define complex conjugation on O to be the [/-invariant map 

v : O -»• O, v(z) = -a(z) (37) 

Now v defines complex conjugation maps 

R(0)-+R(0), f^J and V{0)^V{0), D^D (38) 



where / and D are given by f(z) = f(u(z)) and D(f) = D(f). These complex 
conjugation maps are C-anti-linear M-algebra homomorphisms; in particular fif 2 = 
JJ 2 and = D{D 2 . 

Lemma 2.3.1 

(i) We have E = E and consequently complex conjugation preserves the Euler grad- 
ings of R(0) andV(0) in 

(ii) For x G g, we have f x = —f a (x) and rf = rj a<yX \ 
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Proof: (i) is clear since v is C-anti-linear. For (ii) we note that [x,y] = [o~{x),o~(y)\ 
and (x,y) g = (a(x),a(y)) g for all x, y G g since a is a anti-complex Lie algebra 
involution of g. Consequently 



fx(z) = f x (-a(z)) = -(x,a(z)) s = -(a(x),z) B = -/„(*) (z) 
This computes f x . Using this we find 

rF(fv) = -V x (f*(y)) = -fMy)] = fHx),y] = V a(x) (fy) 

But any vector field on O is uniquely determined by its effect on the functions 
f y G Ri{0). So this proves that rf = r] a ^ . □ 

3 Symbols and Quantization on O 
3.1 Set-up 

If X is a complex algebraic manifold, then the holomorphic cotangent bundle T*X — > 
X is again a complex algebraic manifold. The order d symbol gives a linear map 
V d (X) -> R d (T*X), D ^ where R d (T*X) C i?(T*X) is the subspace of 

functions which are homogeneous (polynomial) functions of degree d on the fibers of 
T*X -> X. We have an algebra grading R(T*X) = ® deZ+ R d (T*X), which we call 
the symbol grading. 

If D G P(JT) has order d, then Sd(D) is the principal symbol of L>, denoted 
by symbol D. See, e.g., ||B-K2| , Appendix] for a resume of some basic facts about 



differential operators and their symbols. 

An algebraic holomorphic action of G on X induces a natural action of G on T*X 
which is algebraic holomorphic symplectic. Then the canonical projection T*X — > X 
is G-equivariant; the induced action of G on T*X is called the canonical lift of the 
G-action on X. The induced representation of G on R(T*X) is locally finite. The 
symbol grading is G-invariant and the principal symbol map is G-equivariant. 

Let A G R(T*0) be the principal symbol of the Euler vector field E on O, i.e, 

A = symbol E (39) 

Then A is G-invariant. The canonical lift of the Euler C*-action on O defines a 
G-invariant algebra grading R(T*0) = ® p& R p (T*0). Then 

R p (T*0) = {<p G R(T*0) | {A, 0} = W>} (40) 

We say that <fi is Euler homogeneous of degree p if <fi G R p (T*0). 
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Now we get a G-invariant complex algebra bigrading 

R(T*0) = R d p {T*0) (41) 

where R d {T*0) = R d {T*0) n R p {T*0). The order d symbol mapping sends V*(0) 
into R d (T*0). 

Let $ x G R(T*0) be the principal symbol of the vector field r] x , i.e., 

$ x = symbol rf for all x G (42) 

Then A, $ a G Rl(T*0). 
For « = 0, . . . , m we set 

fi = f Xi and = (43) 

We recall from ||A-B2|| that these 2m + 2 functions form a set of local etale coordinates 
on O. We also put 

U = fx* ( 44 ) 
We have a Zariski open dense algebraic submanifold O reg of O defined by 

cr» = {zeO | f (z) + 0} (45) 

This construction of O reg breaks the G-symmetry but not the infinitesimal g-symmetry 
by vector fields. By Proposition gXT| , R(0) and V(0) sit inside R(O reg ) and 
V(O re9 ), respectively, as the spaces of g-finite vectors. 

We introduce the following vector fields on O re9 , where i — 1, . . . , m, 

A x > = rj Xi - ^r] X0 and A< = rf< - ^ V x ° (46) 
Jo Jo 

and their principal symbols 

Q Xi = ^ - ^$ x '° and Q< = <&< - ^® x ° (47) 
Jo Jo 

We define a polynomial P{w\, w[, . . . , w m , w' m ) by 

^(adw) 4 (x v ,) = P(wi, w[,..., w m , w' m )x (48) 

where 

m 

w = 2_.( w i x i + wlx^) (49) 
i=i 
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3.2 Main Results 

We assume from now on that g 7^ sp(2n, C), n > 1. We recall 
Theorem 3.2.1 J2EH/ 

(i) There exists a unique (up to scaling) non-zero G-linear map 

r : g -> R\(T*0), x^r x (50) 

So r(g) is i/ie unique subspace in R\(T*0) which is G -irreducible and carries the 
adjoint representation. A lowest weight vector r = r xo inr(g) is given by the formula 

r = i- ( p(e x \ . . . , e Xm , 9*4 - ^a 2 ($ xo ) 2 ] (51) 

Jo V 4 / 

(ii) Suppose p < 4. T/ien i/iere zs no non-zero G-linear map g — > R p _ 1 {T*0). I.e., 
R p _ x (T*0) contains no copy of the adjoint representation. 



Remark 3.2.2 The formula ( |5"T| ) applies equally well when g = sp(2ra, C), n > 1. 
But then P = 0. The symbol /q" ($ x °) 2 easily quantizes to a differential operator on 



O. See A-B2 



Our main result is the G-equivariant quantization of these symbols r x into differ- 
ential operators D x on O in the cases where g is classical. 

Theorem 3.2.3 Assume q is a complex simple Lie algebra of classical type and 
g 7^ sp(2n, C), n> 1. Then there exists a unique differential operator Dq on O such 
that 

(i) Dq has order 4 and the principal symbol of Dq is vq. 

(ii) D is a lowest weight vector in a copy of g inside T>(0). 

(hi) Dq is an Euler homogeneous operator on R(0) of degree —1, i.e., Dq G T>-i(0). 
Moreover we construct Dq as an explicit non-commutative polynomial in the vector 
fields A Xi ,A x 'i, r] x °, and E. We prove that 

A>(/{) = lik)^- 1 (52) 

where j(k) is an explicit degree 4 polynomial in k with leading term k 4 and all coef- 
ficients non-negative. 
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Proof: The uniqueness follows from Theorem |3.2.1 as it says that tq is, up to scaling, 
the unique symbol r G Rz{{T*0) such that r is a lowest weight vector of a copy of 
the adjoint representation. We prove the existence in the next section by case-by- 
case analysis for g = sl(N, C) and g = so(N, C). See Propositions p4. 2. 3| and [4.3.3| . 

□ 

Property (ii) in Theorem 3.2.3| implies that the G-subrepresentation of T>{0) 
generated by Dq is irreducible and carries the adjoint representation. In fact we are 
getting an equivariant quantization in the following sense. 

Corollary 3.2.4 There exists a unique G-linear map 

g^Vi^O), x^D x (53) 

quantizing / |56| j in the sense that, for all x G 0, 

symbol D x = r x (54) 

I.e., D x has order 4 and the principal symbol of D x is r x . Then D Xo is the operator 
Dq constructed in Theorem 3.2.% 

We get two more corollaries to Theorem p.2.3| because of Propositions |2.2.1| and 



2.2.3| . The first is immediate. 

Corollary 3.2.5 The differential operators D x , x G Q, all commute and generate a 
maximal commutative subalgebra A ofV{0). Consequently A = © pe z + -4_ p is graded 
in non-positive degrees with Aq = C and A-i = {D x \ x G q}. 

A is isomorphic to R(0). In fact, the mapping f x i— > D x extends uniquely to a 
G-equivariant complex algebra isomorphism 

R(0) -> A, f^D f (55) 



Corollary 3.2.6 R{0) admits a unique positive-definite Hermitian inner product 
(•|-) invariant under the maximal compact subgroup U of G such that (1|1) = 1 and 
the operators f and Df are adjoint for all f G R(0). Then the formula $2~%) holds. 



Proof: Proposition ^.2.3| applies because Q5^D says that D is non-zero on R P (X) for 



p > 1. This gives us a unique non-degenerate [/-invariant Hermitian inner product 
(•|-) on R(0) such that (1|1) = 1 and / and Df are adjoint. 

We need to check that (-|-) is positive-definite on each space R p (0), p > 1. It 
suffices to check that (/^|/^) is positive for p > 1. Lemma |2.3.1| (ii) implies that 



U = fo (56) 
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since —a{x^) = xq (see Subsection |Q|) . So (p4[) and (|52| ) give 

W) = = DIUD = 7(1) " • -lip) (57) 

which is positive by Theorem 3.2.3| . □ 



Remark 3.2.7 We will show in that D x = -D 



>a{x)- 



3.3 Strategy for quantizing the symbol ro 

Here is our strategy for quantizing the symbol ro into the operator Dq. We start 
from the formula (|5T| ) for the symbol r . This says that S = /o r o where 

S = P(Q X \Q< e x -) - ^\ 2 {<5> xo ) 2 (58) 

Our idea is to construct D by first constructing a suitable quantization S of S which 
is left divisible by / , and then putting D = / _1 S. 

Now 5* is an explicit polynomial (once we compute P) in the symbols Q Xi , Q Xi , A, $> x ° 
of the vector fields A X \A<,E,7] X ° on O reg . Hence S belongs to R(T*O reg ). In fact 
by Theorem |3.2.1| , we know that S belongs to R{T*0). 



The first step of our strategy is to quantize S into a differential operator S on O reg 
using the philosophy of symmetrization from Weyl quantization. This means that we 
obtain S by taking the expression for S, replacing each symbol Q Xi , B% A, $ a: ° by the 
corresponding vector field, and symmetrizing to allow for any order ambiguities. E.g., 
QXiQXi quantizes to ^(A x iA Xl + A^A^). We will refer to this procedure simply as 
'Weyl quantization" . (This does not determine the quantization uniquely, as one can 
symmetrize the symbol [Q x iQ Xi ) 2 in different ways; we merely choose a convenient 
scheme.) 

Thus we construct S as a non-commutative polynomial in the vector fields A Xl , A x *, E, i] x ° 
on O reg . We want to quantize S into S in a G-equivariant way so that S, like S, 
is a lowest weight vector of a copy of ~ g B2 . It turns out to be too hard (and 
unnecessary) to verify all this right away, but we do construct S so that the following 
properties are evident: 

[rf , S] = for all z E g neg and [r?\ S] = -4S (59) 

where 

Qneg = 0-1 © 0-2 (60) 



The conditions in ([59]) go half-way towards proving that S is a lowest weight 
vector of a copy of q u2 . Indeed, we have the following easy fact from the Cartan- 
Weyl theory of representations. 
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Lemma 3.3.1 A vector v in a representation p : g — > EndW is a lowest weight 
vector of a copy ofV^ — Q uk if and only if the following four conditions are satisfied: 

(i) v is Q-finite 

(ii) p h (v) = -2kv 

(iii) p x (v) = for all x G g where g is the orthogonal space to h in go 

(iv) p z {v) = for all z G Q neg 

The second step is to show that f^S belongs to T>(0), (We skip the intermediate 
step of showing that S belongs to T>(0).) In fact, the multiplication operator fo 
commutes with all the vector fields A Xi , A% E, r] x °, and so commutes with S. Thus 
we need not distinguish between / -1 S and S/ _1 . 



We accomplish the second step by implementing Proposition |3.3.4j below. Be- 
fore giving that result, we first prove two more general Lemmas about extending 
differential operators from O reg to O. 

Lemma 3.3.2 Suppose T is a differential operator on O reg . Then T extends to a 
differential operator on O if and only if the operator T : R(O re9 ) — > R(O re9 ) satisfies 
T(R(0)) C R(0). 



Proof: Immediate as O is quasi-affme; see e.g., ||B-K2| , Appendix, A. 4 and A. 6]. □ 



Lemma 3.3.3 Suppose T is a differential operator on O re9 such that [r] z ,T] = 
for all z G Q n eg- Then T extends to a differential operator on O if and only if 
T(f$)eR(0) fork = 0,1,2.... 

Proof: The functions f^, k = 0,1,2..., form a complete set of highest weight 
vectors in R(0) for the g-representation given by the vector fields rj x . Precisely, 
f§ generates Rk(0) under the action of n~ so that U{xc) ■ f§ = R k (0). We have 
n C go © Qneg and also g • ft = C/Jj- by Lemma p.3.1| (iii). It follows that 



R k (0)=U(g neg )-f^ (61) 

Now suppose T(/^) G R{0). Then, since T commutes with rf for all z G g neg , we 
get 

T(R k (0)) = T(U(g neg ) ■ /*) = U( Qneg ) ■ T(/{) C R(0) 



Thus T(R(0)) C R(O) and so T extends to a differential operator by Lemma |3.3.2 . 
The converse is obvious. □ 

To show that / c j" S belongs to V(0) we will use 
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Proposition 3.3.4 Suppose S is a differential operator on O reg such that S is Euler 
homogeneous of degree and S satisfies (\5$). Then the following two properties are 
equivalent: 

(i) f^S g V(0) 

(ii) S(./$) = Jkfof^ 1 fork = 0,1, 2... where 70, 71, . . . are scalars. 
Moreover (i) and (ii) imply 

(iii) / ~ 1 S is a lowest weight vector of a copy of g in V-i(0) . 

Proof: Put T = f$ S. Then [r] z , T] = for all z G Q neg since rf commutes with f 
and S. So Lemma fOPl applies and says that T G £>(£>) if and only if T(/J) G 
for all A; = 0, 1, ... . Hence (ii) implies (i). 

Conversely suppose (i) holds. Then g = T(f^) lies in Rk-i{0) and rj h (g) = 2{k — 
l)g. But it follows using ( |6TD that f^~ l is, up to scaling, the unique eigenfunction in 
R k -i(0) of r] h with eigenvalue 2(fc - 1). So g = 7 Jfe /^~ 1 . This proves (ii). 

Finally we show that properties (i) and (ii) imply (iii). We can apply Lemma 
£31] where = V{0), v = T = / _1 S and fc = 1. Now condition 'i) in Lemma |3.3.1| 



is satisfied since the whole ^-representation T>(0) is locally finite. Also conditions 
(ii) and (iv) follow from ([5^). So we need to check (iii). 

Let x G 0q. We want to show that the differential operator [rf^T] on O is zero. 
We can do this by showing that [^T] annihilates R(0). Now f] x {f\p) = (use 
(H)) and so T(/{) = 7fc/j;~ 1 implies fa*, T](jJ) = 0. But also [rf,T] commutes 



with the vector fields r? 2 , z G Q neg . Indeed \rf , [t7 x ,T]] = [77^, T] = [77*', T] = as 
z' = [z,x] G Qneg- So then [rf^T] annihilates R{0) because of ([TO). □ 

In carrying out this plan for quantizing ro, we encountered a problem as we found 
no "Weyl quantization" of the term A 2 ($ x '°) 2 in (^3) which produced an operator S 
satisfying the condition S(/^) = 7fc/o/^ _1 - (Such a Weyl quantization may or may 
not be possible here; our 'Weyl quantization" of the first term is already not unique. 
Probably Weyl quantization fails to give the operators we want; cf. the quantization 



m 



E3 



To get around this, we applied Weyl quantization to the first term in flog ) after 
computing P explicitly. Then we "guessed" that \ 2 (Q X °) 2 should quantize into the 
operator 

(E + Cl )(E + c 2 )(rf ) 2 (62) 
where c\ and C2 were unknown constants. There is no problem in this as 

Lemma 3.3.5 For any polynomial q(E) in the Euler vector field, the operator q(E)(rj 
is a lowest weight vector of a copy ofV^ ~ g H2 in V{0). 



x \2 
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Thus we got a candidate for S with unknown parameters c\ and c 2 . We then 
calculated S(/^) and "solved" for c\ and c 2 by demanding that S(fh) be a multiple 
of /o/^ _1 - There was no guarantee a priori that solutions for c\ and c 2 existed, but 
in fact we found a unique solution for the polynomial {E + C\)(E + c 2 ). 

In our presentation below, we in fact use a general polynomial q(E) in place of 
the particular polynomial (E + C\)(E + c 2 ). We show that the required relation 
— Ikfoftp -1 forces g(-E) to be of the form (E + C\){E + C2) and we determine 
C\ and c 2 . 

In working this out, we do not yet have a unified treatment of all complex simple 
Lie algebras. Instead we treat only the cases where g is classical. We work out the 
polynomial P and the operator S in Section |4], first for g = sl(N, C) and then for 
g = so(N,C)- 

We conjecture that Theorem |3.2.3| holds also for the five exceptional simple com- 
plex Lie algebras. 



4 Explicit construction of Dq for q classical 



4.1 Coordinate representations of vector fields 

Our aim is to construct -Do as the quantization of the symbol r . Our strategy, 
developed in Subsection |37J|, is to quantize the symbol S = f r into an operator S 
which satisfies condition (ii) in Prop |3.3~4| To construct S and compute S(/jj), we 
will simply work out everything in terms of our local coordinates fo, ft, fi, ft on O 
from ©. 

Fortunately, in proving Theorem |3.2.1| in |[A-B2|| , we first found a formula for the 
highest weight vector f$ G R\{0) ~ g. This was: 

Proposition 4.1.1 [|A-B2| The unique expression for the function f^ e R±(0) in 
terms of our local coordinates ft, ft, fi, f[, i = 1, . . 

fi> = ~7§P(fh fii • • • j /lj fi) 
Jo 



,m, on O is 
fo) 



1 (ft" 2 



4 /, 



(63) 







The unique expressions for the vector fields rj Xi , if*, rf° in terms of our local 

1 m, 



coordinates fi, ft, f , ft are, where i 

V Xi = -fo^jj + fi^jj V' =Jo— + ^7777, V 

These follow as rj x (f y ) = f^ x ^ for any x, y 6 g. Then ([46]) gives 

x d d 

dfi dft 



f fL 4. fiL 
J °dfi Ji df^ 



.Co 



2f 9 



(64) 



(65) 
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The operators A Xl , A x * , r] x ° also span a Heisenberg Lie algebra since 

[A x 'i, A x 'i] = [A Xl ,A Xj ] = [A x 'i, rf°\ = [A x \rf°\ = 0, [A x \ A x 'j] = 25 ij rf° (66) 

The next two facts are trivial to verify. 

Lemma 4.1.2 The operators A X \A X '^ all commute with the vector fields rf where z 
lies in the Heisenberg Lie algebra Q neg - 

Lemma 4.1.3 We have the relations r] h (f ) = —2f , r) h (fo) = 0; V h (fi) = ~fi> 
V h (fl) = -flfori = l,...,m, and fa", A*] = -A» ifyEQ-i. 



Also, we have rj 



f[x ,x^] = f-x' Q = -fo and SO Usin S (11) We find 



(v X0 ) 2 f!p = [-ZkfoU + k(k - l)(/o) 2 ] /, 



k-2 



(67) 



To reduce the number of formulas, we introduce the superscript e with e = ±1 
where e = 1 indicates primed quantities while e = — 1 indicates unprimed quantities. 
Thus x\ = x\ if e = 1 while x\ = Xi if e = — 1 and so on. Then fl64]) and (|65| ) give 



t 9 te d 



A 1 



ef l f ed_ 



(6* 



4.2 The case q = 8l(n+ 1,C), n > 2 

In this case g is of type A n . The minimal nilpotent orbit O is 

= {X G | X 2 = 0, rank(X) = 1} 
Then dim O = 2n and so 

m = n — 1 



(69) 



(70) 



A standard basis of gl(n+ 1, C) is given by the elementary matrices Eij, < i, j < n. 
We choose x^ = E 0jn , x' Q = E 0<0 — E n>n , and x = E n Q . Then the normalized Killing 
form on q is given by (X,X') S = |(TraceXX'). We take the Cartan involution a to 
be a(X) = -X*. 

The matrices x' p = E n>p and x p = E Pj o, p = 1, . . . ,m, form a basis of g_i and 
satisfy the conditions in (p3|). So we get 



/ 

w x 



w = ^(vjpEpfl + w' p E n , p ) 
P =i 



\ 




... 
\ w[ ... w' m J 



(71) 
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Next we need to compute the polynomial P denned in (pEB|). We have the standard 
operator identity Ad(exptw) = J2T=o F( a d^) fc and so fl48|) gives 

P(wi, w'j) is the coefficient of t A xo in (Ad(exptiy)) • x^ (72) 

We can easily compute this since 

(Ad(exptw)) • x^p = (exp tw)x^,(exp — tw) (73) 

where the right side is a matrix product. We find 

exp(tw) = / + tw + -t 2 (J2™ =1 w p w'^ E nfl (74) 

since w 2 = (X)^=i w p w p )E n ,o and u; 3 = 0. Then we get 

If \ 2 
P(w h w 1 ,...,w m ,w m ) = - [T^i w P w' p J (75) 

Now plugging this expression for P into (|63"D and([51|) we get 

/* = 7^3 F - (Mo) 2 ] and r = -L[A 2 -A 2 ($^) 2 ] (76) 
4 Jo 4 io 

where 

m m 

a = Y t f i f i and A = ^0^ (77) 

»=i i=i 

To begin the process of quantizing S = for , we quantize A into the operator 



1 m 

A = -^(A Xl A^ + A 3 * A**) (7? 



2 

i=i 

Lemma 4.2.1 A is a differential operator on O reg of order 2 with principal symbol 
A. A is Euler homogeneous of degree 0. A satisfies [r/ z , A] = for all z G g neg and 
[r/\A] = -2A. 

Proof: The first statement is clear. The commutativity [77*, A] = follows by 
Lemma |4.1.2| and the weight relation [77 , A] = — 2A follows by Lemma |4.1.3| . □ 



By calculation, we find the following two formulas, where the answer is expressed 



in terms of our "e" notation defined at the end of Subsection 4.1 



A*i(a)=e/ tf and A** (/„/£) = -fofl (79) 
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So we get 



2 ft 

Next we determine A(f^) starting from the identity 

m 

A(4) = k(AU)f^ + k(k - 1)4" 2 ^(A^)(A*^) 

i=l 

Using © and © we find A*'(/^)A<(/^) = -fif'Mh and so 



£(A^)(A^) 



' /o 



i=l 

Now we need the first term in (|81|). First (|80|) and flT9|) give 

A*tA*;\U) = — + e/o/ ° ~ 2/ '^ 

2/o 

and consequently 

h A ^A< + A<A^)(U) = - a + 2 /^ 
Now summing over i = l,...,mwe get 



A(^) = -(l + f)f 
Jo 



Now substituting into (pTT) we get 



A(/*) = -*(* + 



«/, 



fc-i 



/o 



We need to calculate A 2 (/^). Since A commutes with multiplication by 
power of /o, (0) gives 



A 2 (4) = -A;(fc + f)/ - 1 A(a4- 1 ) 
So the problem is to compute 

A(af k f l ) = (Aa)f!;- l + a(Af^) 

m 

+ (k - l)4" 2 ^(A^a)(A^^) + (A^)(A^ ( 
i=i 
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Easily A (a) = —mf^ and also 



(A** a) (A** U) 



and (|79|) give 

fifi 



2/0 



a + efofd) 



So 



19) 



a) = — t 



i=l 



of- 

A, 



(90) 



Collecting all these terms we find 



Thus 



gives 

A 2 (4) = 



k-\\ 



fl2 -k-2 



-mfSf}- 1 - (k - I) (k+ f ) -/, 

Jo 



,2 1 



mk (k + f ) f U + k(k -l)(k + f ) 2 -^ 

Jo . 



fc-2 



(91) 



(92) 



Our aim is to find a polynomial q{E) in the Euler vector field E such that S = 
I (A 2 — q(E)(rj x °) 2 ) satisfies S(/^) = afc/o/^ -1 for some scalars A priori, there 
is no guarantee that such an operator q(E) exists. But fortunately ([76]) gives the 
relation 



a 



i-2 (/o) 
Jo 



4/o 



(93) 



A2 



This is the unique linear relation in R(O re9 ) between the three functions jj, (f' Q 
and fof^p. We have q(E)(f§) = q(k)f§. Now comparing (|92^) with (167|) we conclude 



that g(-E) exists if and only if k(k — 1) (k + 



m\2 



g(fc)fc(fc-l). Thus g(fc) = (fc + 



m\2 



is the unique solution. Then arithmetic gives a formula for a>k (which turns out to 
simplify very nicely). This proves 

Lemma 4.2.2 Let S = \ (A 2 - q(E)(f] x °) 2 ) where q{E) is some polynomial in the 
Euler operator E. Then S is a differential operator on O reg of order 4 with principal 
symbol S. S is Euler homogeneous of degree and satisfies the conditions in ([5£ 
The polynomial 



q(E) = (E + f)' 



(94) 



is the unique polynomial in E such that S satisfies (ii) in Proposition \3.3. j| . Then 
S(/{) = aMf 1 where 



a k = k\k + ^)(k + ^) 



(95) 
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The polynomial oik has leading term k 4 and all coefficients non-negative since 
m > 1. Proposition |3.3.4| now says that / ~ 1 S is a lowest weight vector of a copy of 



g in U_i(0) Thus we have proven 

Proposition 4.2.3 Let g = sl(m + 1,C) ; m > 1, and define A by ffTfy). The 

differential operator 

Do = |S = -L [A 2 - (E + f ) 2 (rf °) 2 ] (96) 
Jo 4 Jo 

on CT 69 extends to an algebraic differential operator on O. Dq satisfies all three 
conditions in Theorem \3. 2. 3j . We have 

A>(/{) = akft 1 (97) 

where is given by 



Remark 4.2.4 We can show that A extends to a differential operator on O. 
4.3 The case g = so(N, C), N > 6 

Here g is of type D n UN = 2n is even or of type B n if N = 2n+ 1 is odd. We unify our 
treatment of the two types by using the obvious inclusion so(2n, C) C so(2n + 1, C). 



The "final answer" given in Proposition 4.3.3 below is stated in a uniform way. 



To begin with, we introduce a convenient model for the complex Lie algebra 
g = 5o(N, C). Let u-v be & symmetric non-degenerate complex bilinear form on C . 
We have a complex linear Lie bracket on A 2 C N given by 

[a A 6, c A d] = (a ■ c)b A d + (6 • d)a A c - (a • d)b A c - (6 • c)a A d (98) 

Each vector u A v defines a skew-symmetric linear transformation L uAv on C n by 

£uAi>( a ) = ( u " a ) w ~ ( v " a ) w (99) 

Extending linearly, we obtain a natural complex Lie algebra isomorphism A 2 C N —* 
so(N, C), z ^ L z . We use this to identify g with A 2 C N . 
Now the minimal nilpotent orbit O is 

O = {u Av e g = A 2 C N \u-u = u-v = vv = 0, u A v ^ 0} (100) 

The complex dimension of O is dim O = 2N — 6, and so 

m = N-A (101) 
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Thus m = 2n — 4 if g has type D n , while m = 2n — 3 if g has type B n . 

We next fix a convenient basis of C^. If N = 2n + 1, we let t>o denote a vector 
in C w such that Vq ■ Vq = 1; if iV = 2n, we set uq = 0. Then we have a direct sum 
decomposition = V © Cw where is the orthogonal space to v . We fix a basis 

v[, . . . , v n , v' n oiV such that u 4 • = i>| • i> ' = 0, and ■ v'j = 5^. This basis of V, 
together with v if iV is odd, is a basis of C . 

Now we choose 

x = -v n - 1 Av n , x^ = v' n _ 1 Av' n , x' = v n -i A v' n _ x + v n Av' n (102) 
The normalized Killing form on g is given by 

(a A b, c A d) g = - [{a ■ c){b ■ d) - (a • d)(6 • c)\ (103) 

We take the Cartan involution a of q to be the one induced in the natural way from 
the C-anti-linear involution a : C N —> C N such that cr(fj) = v[ and a(v ) = v . 

We can now construct a nice basis of This basis is given by the set u| A 
v n -i,vf A f n where i = 1, . . . ,n — 2, e = ±1, together with t> A f n _i, t>o A t> n if iV is 
odd. We label the basis vectors in the following way: 

Xo = v At> n -i, x' = v Av n (104) 

and for i = 1, . . . , n — 2, 

X; = t>j A w n _i, iEi = A x- = w-Aw„, = f, A t) n (105) 

The bracket relations among basis vectors are all zero except for the following ones 

[x'^Xj] = [x'^Xj] = 5ijX , [x ,xo\ = x (106) 

where i,j G {1, . . . , n — 2}. 

Referring back to Subsection |2.3| , we see now that all the relations in ([£J) are 
satisfied if we take xi, . . . , x n _ 2 , x ,xi, . . . , x n _ 2 and x[, . . . , x' n _ 2 , x' , x[, . . . , x' n _ 2 to 
be, respectively, the lists Xi, . . . ,x m and x[, . . . ,x' m . (Here we ignore x and x' if 
N is even.) We have broken the sets x%, . . . ,x m and x'±, . . . , x' m into these peculiar 
groups because this is necessary to write down P in the next step. 

Now we put 



n-2 

id = w x + w' x' + 'S^{w i x i + WiXi + w' i x\ + w'iX'j) (107) 

8=1 
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Then by writing out w in matrix form and computing as in (|72|) and (|73|) we find 
the following formula for P: 

P(w i} Wi, w' u w[, w , w' Q ) =\ ((J27=i W M - &iWi) - w o w' ) 2 

(108) 



Then plugging this expression for P into (p3|) we get 

^ = J73( a2 + 4&c -(/o/o) 2 )- (109) 

where 

a = (SILl 2 fif'i ~ fifi) ~ fofo 

b = (EL -2 hi) + |(7o) 2 (no) 

c=(Er= 2 /^)-K/o) 2 

By making the substitutions, where i — 1, . . . , to, 

//^e* 1 , £ e ^©% ^^e^o 

we get symbols A, £?, C corresponding to the functions a, b, c. Then plugging our 
expression for P into (HI) we get 

r = _L (A 2 + ABC - A 2 ($^) 2 ) (111) 

The Weyl quantization of these principal symbols A,B,C are the operators 
A =\ {YI=i A< + A x '*A Xi - A 2i A 5 * - A z *A 2i ) 

(112) 

_i (A xo A x o + A^A^ ) 



B = (J27=i A*<A*i) + ±(A*°) 2 (113) 

C = (Er= 2 A 5 ^A^) - |(A 2 o)2 (H4) 

Lemma 4.3.1 A ; B and C are differential operators on O reg of order 2 with respec- 
tive principal symbols A, B, C . A, B and C are all Euler homogeneous of degree 
and each one commutes with the vector fields rj z , z £ Q n eg- We have [rj h , A] = — 2A ; 
[V H ,B] = -2B, and [r/\ C] = -2C. 
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Proof: Same as for Lemma 4.2.1 . □ 

Now routine calculation results in the formulas 



A(4) 



k p Jib 

Jo 



B(4) 



-tXfV c (4) = fir 1 ( 115 ) 

Jo Jo 



where 



t k = k{k + — -l) 



(116) 



Next more computation gives the following formulas, where we have set r k 
fc + f-2. 



Au fc )-((^-i) fl2+4 7 rfca2 +wo)wr 



(117) 



2CB(4) = I (k - 1) 



a 2 + Abe + 2r k bc + af Q f Q 
To 2 



+ mfofo t k f; 



e k-2 



;ii8) 



2BC(/{) = l(k - 1) 



, a 2 + 46c + 2r fc 6c - aft/ 



/o 2 



Now combining the last three calculations we get 
(A 2 + 2BC + 2CB)(4) 



3mt k foU + (k-l)(k + ^ + l)t k 



ek-2 



a 2 + 46c 

~J3~. 



fc-2 



(119) 



(120) 



Our aim is again to find q(E) so that we can build the operator S with the desired 
properties. Proceeding as in the previous subsection, we compare (II20D with 
By ( |109| ) we have the relation 



a 2 + Abe 



- (foY = *foU 



(121) 



This is the unique linear relation in R(O re9 ) between the functions a2 ^ 4bc , (/o) 2 , and 
/oJV As in the previous subsection, we obtain 
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Lemma 4.3.2 Let 



S = - ((A 2 + 2BC + 2CB) - q(E)(7] X0 ) 2 ) 



(122) 



where q(E) is some polynomial in the Euler operator E. Then S is a differential 
operator on O reg of order 4 with principal symbol S . S is homogeneous of degree 
and satisfies the conditions in ([5$). The polynomial 

q{E) = (E+& + l){E + f-l) (123) 

is the unique polynomial in E such that S satisfies (Hi) in Proposition 3. 3.4 - Then 

'AO 



S(/{) = Moft 1 where 



771 771 I 

p k = k(k + i)(k+--i)(k+---: 



(124) 



The polynomial (3k has leading term k 4 and all coefficients non-negative since 
m > 2. Proposition |3.3.4j now says that / _1 S i s a l° wes t weight vector of a copy of 
q in T>_i(0) Thus we have proven 

Proposition 4.3.3 Let g = so(m + 4, C) ; m > 2. The differential operator on O reg 

1 



^o = T S 

Jo 



(125) 



defined by (\122j and (\123$ extends to an algebraic differential operator on O. D 
satisfies all three conditions in Theorem \S.2.$ . We have 



(126) 



where (3^ is given by ( U]W 



Remark 4.3.4 We can show that A, B and C extend to differential operators on 
O. 



5 Relations to other work 

We return to the more general situation of i^-orbits on p discussed in the introduc- 



tion. The invariant theory of the action of K on p was analyzed in [[K-R ]. 

Suppose g is a simple Lie algebra and t has non-trivial center. Then the action of 
the center of K defines a i^-invariant splitting p = p + ©p~. Let Y be a i^-orbit inside 
p + . Then T>(Y) has been studied for various cases of this sort in ||L-Sm|| , ||L-Sm-St| , 
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[ L-St | where the functions and vector fields on Y do not generate T>(Y). They 
construct an extension of the natural infinitesimal action t — > QJectF to a Lie algebra 
homomorphism tt : g — > T>(Y) where p + acts by multiplication operators and p~ acts 
by commuting order 2 differential operators on Y which are Euler homogeneous of 
degree —1. 



In particular, in | L-Sm-St |, Levasseur, Smith and Stafford analyzed cases where 

Y = O m i n PI p + and O m i n is the minimal nilpotent orbit of g. They prove that the 
kernel of the algebra homomorphism U(g) — > T>(Y) defined by their map tt is the 
Joseph ideal. In ( ||L-St|| ) Levasseur and Stafford analyzed T>{Y) where Y is any K- 
orbit in p + and g is classical. Using the theory of Howe pairs and the Oscillator 
representation, they construct tt for Y "sufficiently small" and prove that the kernel 
of the corresponding algebra homomorphism U(g) — > T>(Y) is a completely prime 
maximal ideal. 

Now suppose g is simple and 6 has trivial center. Then p is irreducible as a 
t-module. In [|B-K2|| , R. Brylinski and B. Kostant studied cases of this sort where 

Y = O m i n H p. They constructed a space L of commuting differential operators on 

Y such that L is G- irreducible and carries the i^-representation p. The operators in 
L have order 4 and are Euler homogeneous of degree —1. The highest weight vector 
in L was constructed as the quotient by a multiplication operator of a homogeneous 
quartic polynomial in certain commuting vector fields of the 6-action. 

Our methods are an extension of that construction. However, our quantization 
is considerably more subtle as the our principal symbol r$ involves a homogeneous 
quartic polynomial in the symbols of non-commuting vector fields (which lie outside 
the t-action). The quantization then requires not only symmetrization but also the 
introduction of lower order correction terms. Theorem \5 . 2 . 3| says that these correction 
terms are uniquely determined. 

In [|L-0|1 , Lecomte and Ovsienko show that the algebra of polynomial symbols on 
]R n admits a unique quantization (involving non-obvious lower order terms) equiv- 
ariant under the vector field action of sl(n + 1, K) on R™ which arises by embedding 
M. n as the "big cell" in MP n . It would be interesting to see if there is a g-equivariant 
quantization map Q : R{T*0) -> V(0). 
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